Impact of automation: measurement of performance, workload and behaviour in a complex control environment

This paper describes an experiment that was undertaken to compare three levels of automation in rail signalling; a high level in which an automated agent set routes for trains using timetable information, a medium level in which trains were routed along pre-defined paths, and a low level where the operator (signaller) was responsible for the movement of all trains. These levels are described in terms of a Rail Automation Model based on previous automation theory (Parasuraman et al., 2000). Performance, subjective workload, and signaller activity were measured for each level of automation running under both normal operating conditions and abnormal, or disrupted, conditions. The results indicate that perceived workload, during both normal and disrupted phases of the experiment, decreased as the level of automation increased and performance was most consistent (i.e. showed the least variation between participants) with the highest level of automation. The results give a strong case in favour of automation, particularly in terms of demonstrating the potential for automation to reduce workload, but also suggest much benefit can achieved from a mid-level of automation potentially at a lower cost and complexity

Reducing Uncertainty in PHM by Accounting for Human Factors - A Case Study in the Biopharmaceutical Industry

The ultimate goal of prognostics within Through-life Engineering Services (TES) is to accurately predict the remaining useful life (RUL) of components. Prognostic frameworks inherently presume that there is predictability in the failure rate of the system, i.e. a system experiencing exclusively stochastic failure events cannot, by definition, be predictable. Prediction model uncertainties must be bound in some logical way. Therefore, to achieve an accurate prognostic model, uncertainty must first be reduced through the identification and elimination of the root causes of random failure events. This research investigates human error in maintenance activities as a major cause of random failure events, using a case study from the biopharmaceutical industry. Elastomer failures remain the number one contamination risk in this industry and data shows unexplained variability in the lifetime of real components when compared to accelerated lifetime testing in the lab environment. Technician error during installation and maintenance activities of elastomers is one possible cause for this and this research explores how these errors can be eliminated, reduced, or accounted for within the reliability modeling process. The initial approach followed was to improve technician training in order to reduce errors and thereby reduce the variability of random failure events. Subsequent data has shown an improvement in key metrics with failures now more closely matching data from lab testing. However, there is scope for further improvements and future research will explore the role of performance influencing factors in the maintenance task to identify additional causes of variation. These factors may then be incorporated as a process variable in a prognostics and health management (PHM) model developed for the system. The paper will present these data fusion approaches accounting for human factors as a roadmap to improving PHM model reliability

Estimation of Train Driver Workload: Extracting Taskload Measures from On-Train-Data-Recorders

This paper presents a method to extract train driver taskload from downloads of on-train-data-recorders (OTDR). OTDR are in widespread use for the purposes of condition monitoring of trains, but they may also have applications in operations monitoring and management. Evaluation of train driver workload is one such application. The paper describes the type of data held in OTDR recordings and how they can be transformed into driver actions throughout a journey. Example data from 16 commuter journeys are presented, which highlights the increased taskload during arrival at stations. Finally, the possibilities and limitations of the data are discussed

Safety Risk Registers: Challenges and Guidance

A risk database, or risk register, is a central tool for organisations to use to monitor and reduce risks, both those identified during initial safety assessments and those emerging during operations (Whipple and Pitblado, 2010). The risk register should contain all analysed risks and should prioritise the areas that require managerial attention. When populated with information on each risk, including risk ranking, the risk register can be analysed to present the risk profile for different aspects of the organisation (Filippin and Dreher, 2004). When reviewed and updated over time, it can also be analysed to present trends within the risk profile and focus management attention on the highest risk activities or facilities (Whipple and Pitblado, 2010). In order to successfully develop a risk registry that provides an accurate level of risk within a process, there is a requirement for real time data on risk to be input into a risk registry. Despite their place at the heart of safety management, there is relatively little guidance and research on how to construct, maintain and use a risk register. The challenges and ideas in this paper were developed during the initial phase of a case study to develop a single central risk register for an energy generation company. The case study used workshops with key stakeholders from the company to work through the issues faced in developing a single integrated risk register. Challenges faced ranged from ensuring employees contributing to the risk register had a basic understanding of risk concepts, through identification and scope of hazards to be included, to data collection and automatic population of the risk register. The challenges encountered during this project are believed to be those that many companies face, and therefore the resolutions proposed and adopted for this case study will be presented here as guidance for the implementation and management of risk registries in safety management systems

Total Safety Management: What Are the Main Concerns in the Integration of Best Available Methods and Tools

“BP released findings from its own internal investigation of the oil spill in the Gulf of Mexico, revealed inefficient Organization interfaces among BP management, the rig crew and well site leader. Also according to the investigation, one important contributor to the accident was inadequate guidelines for critical tests and operations”. (Pires and Mosleh 2011) Over the recent past, the accumulation of major mishaps, crises and accidents have made it clear that organisations must still improve their capabilities to address safety “not as a stand-alone activity that is separate from the main activities and processes of the organization” but as an integrated part of total performance management. The requirements for safety management in existing and upcoming standards and regulations, as for example the ISO 31000 and or the Seveso II directive, call for a proactive strategic approach, anticipating risks and demonstrating a capacity to keep safety at the centre of changes driven by commercial competition, and ensuring that safety evidence itself becomes an effective driver of change. However there is often a gap between the state principles and an actual roadmap to their implementation. Furthermore organisations, especially the one dealing with safety critical operations, find it difficult to integrate their different functional units in a common programme of operations management or change; there is no clear consensus about what it means to be ‘proactive’; there is no integrated framework for analysing or managing all the human related functions in an operational system. Innovation may rely on assembling the best practices, tools and methods already available for functional analysis, risk assessment, interactive emergency scenarios analysis, performance monitoring, design review, training and knowledge management, in an integrated framework able to address safety management in the main aspects of a product or process lifecycle the cornerstone of which is the building of a common operational picture to support the capacity to perform more participatory and dynamic risk identification and solutions loops in: - Design (new plants, processes /procedures availing new visualization tools) - Ad hoc critical activities (management of change or scheduled overhaul) - Operations management (establishing of dynamic risk registers). This is the scope of a new EU funded research project called TOSCA and the present paper will introduce the current framework being built

Physiological Measurements for Real-time Fatigue Monitoring in Train Drivers: Review of the State of the Art and Reframing the Problem

The impact of fatigue on train drivers is one of the most important safety-critical issues in rail. It affects drivers’ performance, significantly contributing to railway incidents and accidents. To address the issue of real-time fatigue detection in drivers, most reliable and applicable psychophysiological indicators of fatigue need to be identified. Hence, this paper aims to examine and present the current state of the art in physiological measures for real-time fatigue monitoring that could be applied in the train driving context. Three groups of such measures are identified: EEG, eye-tracking and heart-rate measures. This is the first paper to provide the analysis and review of these measures together on a granular level, focusing on specific variables. Their potential application to monitoring train driver fatigue is discussed in respective sections. A summary of all variables, key findings and issues across these measures is provided. An alternative reconceptualization of the problem is proposed, shifting the focus from the concept of fatigue to that of attention. Several arguments are put forward in support of attention as a better-defined construct, more predictive of performance decrements than fatigue, with serious ramifications on human safety. Proposed reframing of the problem coupled with the detailed presentation of findings for specific relevant variables can serve as a guideline for future empirical research, which is needed in this field

Understanding is key: an analysis of factors pertaining to trust in a real-world automation system

Objective: This paper aims to explore the role of factors pertaining to trust in real-world automation systems through the application of observational methods in a case study from the railway sector.Background: Trust in automation is widely acknowledged as an important mediator of automation use, but the majority of the research on automation trust is based on laboratory work. In contrast, this work explored trust in a real-world setting.Method: Experienced rail operators in four signalling centers were observed for 90 min, and their activities were coded into five mutually exclusive categories. Their observed activities were analyzed in relation to their reported trust levels, collected via a questionnaire.Results: The results showed clear differences in activity, even when circumstances on the workstations were very similar, and significant differences in some trust dimensions were found between groups exhibiting different levels of intervention and time not involved with signaling.Conclusion: Although the empirical, lab-based studies in the literature have consistently found that reliability and competence of the automation are the most important aspects of trust development, understanding of the automation emerged as the strongest dimension in this study. The implications are that development and maintenance of trust in real-world, safety-critical automation systems may be distinct from artificial laboratory automation.Application: The findings have important implications for emerging automation concepts in diverse industries including highly automated vehicles and Internet of things

Appropriate automation of rail signalling systems : a human factors study

This thesis examines the effect of automation in the rail signalling environment. The level of automation in a system can be described as ranging along a continuum from manual control to fully autonomous automation and development of appropriate automation for a system is likely to enhance overall system performance. Network Rail, the company which owns, operates, and maintains the rail infrastructure in the UK, envisions increasing levels of automation in future rail systems, but prior to this research, little structured evaluation of current automation had been undertaken. The research performed for this thesis set out to examine the impact of automation on rail signalling. A rail automation model was developed to illustrate the levels of automation present in different generations of signalling system. The research focussed on one system in particular, the Automatic Routing System (ARS). The ARS has been present in modern signalling centres since the late 1980s. It uses timetable information to set routes for trains arriving on its area of control and incorporates complex algorithms to resolve conflicts between trains. Multiple methods were used to investigate current signalling automation. An understanding of the signalling domain underpinned the research, and a model was developed to illustrate the type and level of automation present in different generations of current signalling systems. Structured observations were employed to investigate differences in activity between individual signallers. As a part of this study, a relationship was found between observed intervention levels and some of the trust dimensions identified from the literature. A video archive analysis gave initial insight into some of the issues signallers had with automation, and semi-structured interviews carried out with signallers at their workstations built on these themes. The interviews investigated four areas; signallers’ opinions of ARS, system performance issues, knowledge of ARS, and interaction with ARS. Data were gathered on a wide variety of individual issues, for example on different monitoring strategies employed, interaction preferences, signallers’ understanding of the system and their ability to predict it. Data on specific issues with ARS also emerged from the interviews, for example the impact of poor programming and planning data, and the poor competence of the system, particularly during disruption. An experiment was performed to investigate the differences between different levels of automation under both normal and disrupted running. The experiment gathered quantitative data on the effect of different levels of automation on workload and performance in addition to eye tracking data which were used to gain insight into signaller monitoring strategies. The results indicate that ARS does reduce workload and increase performance, and it does so in spite of deficiencies in terms of feedback to the signaller. This lack of feedback makes it difficult for the signaller to understand and predict the automation and, hence, creates difficulties for the operator. In addition, the methods for controlling ARS are limited and it can be difficult for the signallers to work cooperatively with the system. Principles of good automation were identified from the literature and recommendations based on these and the findings of the research were developed for future signalling automation systems. These highlighted the importance of improving feedback from ARS and the ability of the signaller to direct the system. It is anticipated that these improvements would allow the signaller and the automation to work more closely together in order to maximise overall system performance. The principles of automation are intended as a generic guidance tool and their application is not confined to rail signalling. There may also be wider implications from the research such as the influence of operators’ ability to understand and predict automation in automation use, and the existence of different types of monitoring behaviour.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Appropriate automation of rail signalling systems: a human factors study

This thesis examines the effect of automation in the rail signalling environment. The level of automation in a system can be described as ranging along a continuum from manual control to fully autonomous automation and development of appropriate automation for a system is likely to enhance overall system performance. Network Rail, the company which owns, operates, and maintains the rail infrastructure in the UK, envisions increasing levels of automation in future rail systems, but prior to this research, little structured evaluation of current automation had been undertaken. The research performed for this thesis set out to examine the impact of automation on rail signalling. A rail automation model was developed to illustrate the levels of automation present in different generations of signalling system. The research focussed on one system in particular, the Automatic Routing System (ARS). The ARS has been present in modern signalling centres since the late 1980s. It uses timetable information to set routes for trains arriving on its area of control and incorporates complex algorithms to resolve conflicts between trains. Multiple methods were used to investigate current signalling automation. An understanding of the signalling domain underpinned the research, and a model was developed to illustrate the type and level of automation present in different generations of current signalling systems. Structured observations were employed to investigate differences in activity between individual signallers. As a part of this study, a relationship was found between observed intervention levels and some of the trust dimensions identified from the literature. A video archive analysis gave initial insight into some of the issues signallers had with automation, and semi-structured interviews carried out with signallers at their workstations built on these themes. The interviews investigated four areas; signallers’ opinions of ARS, system performance issues, knowledge of ARS, and interaction with ARS. Data were gathered on a wide variety of individual issues, for example on different monitoring strategies employed, interaction preferences, signallers’ understanding of the system and their ability to predict it. Data on specific issues with ARS also emerged from the interviews, for example the impact of poor programming and planning data, and the poor competence of the system, particularly during disruption. An experiment was performed to investigate the differences between different levels of automation under both normal and disrupted running. The experiment gathered quantitative data on the effect of different levels of automation on workload and performance in addition to eye tracking data which were used to gain insight into signaller monitoring strategies. The results indicate that ARS does reduce workload and increase performance, and it does so in spite of deficiencies in terms of feedback to the signaller. This lack of feedback makes it difficult for the signaller to understand and predict the automation and, hence, creates difficulties for the operator. In addition, the methods for controlling ARS are limited and it can be difficult for the signallers to work cooperatively with the system. Principles of good automation were identified from the literature and recommendations based on these and the findings of the research were developed for future signalling automation systems. These highlighted the importance of improving feedback from ARS and the ability of the signaller to direct the system. It is anticipated that these improvements would allow the signaller and the automation to work more closely together in order to maximise overall system performance. The principles of automation are intended as a generic guidance tool and their application is not confined to rail signalling. There may also be wider implications from the research such as the influence of operators’ ability to understand and predict automation in automation use, and the existence of different types of monitoring behaviour